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"VIDEO DECODING METHOD AND DEVICE" 



FIELD OF THE INVENTION 

The present invention generally relates to the field of video cajmpression and, more 
particularly, to a video decoding method for the decompression of a coded bitstream 
corresponding to an original video sequence that has been divided into successive groups of 
frames (GOft) and coded by means of a 3D subband video coding method comprising the 
following steps : 

- a temporal filtering step - with or without motion compensation - performed on 
each successive couple of frames In each GOP of saki sequence ; 

- a spatial analysis step, perfomied on said filtered sequence ; 

- an entropy coding step, perfonmed on said analyzed filtered sequence, and on 
motion vectors in case of motion compensation ; 

- an arithmetic coding step, applied to the coded sequence thus obtained and 
delivering said coded iM'tstream. 

The invention also relates to a decoding device for carrying out said decoding 
method, to a memory medium including a code for perfonnliiQ the steps of said decoding 
method, and to a corresponding apparatus. 

BACKGROUND OF THE liWENTION 

From MPEG-1 to H.26L, standard video compression schemes were based on so- 
called hybrid solutions : an hybrid video encoder uses a predictive scheme where each frame of 
the input video sequence is temporally predicted from a given reference frame, and the 
prediction error thus obtained by difference between said frame and its prediction is spatially 
transformed (the transfomi is for instance a bi-dimensional DCT transform) in orcler to get 
advantege of spatial redundancies. 

A different approach, later proposed, consists In processing a group of frames 
(GOF) as a three-dimensional (3D, or 2D + 1) structure and spaHo-temporally filtering it in order 
to compact the energy in the low frequencies (as described for instance in Three-dimensional 
subband coding of video", Cl. Podilchuk and al., IEEE Transactions on Image Processing, vol.4, 
n«»2, Febnjary 1995, pp.125-139). The introduction of a nwtion comp^sation step In such a 3D 
subband decomposition scheme allows to Improve the overall coding eflfidency and leads to a 
spatio-temporal multiresolutlon (hierarchical) representation of the video signal thanks to a 
subband tree, as depicted in Rg.l. 

The illustrated 3D wavelet decomposition with motion compensation is similarly 
applied to successive groups of frames (GOFs). As shown in Fig.l, each GOF of the input video, 
including in the illustrated case eight frames Fl to F8, is firet motion-compensated (MQ In order 
to process sequences with large motion, and then temporally filtered (TF) using Haar wavelets 



^^FR020065 Epp 

(the dotted arrows correspond to a high-pass temporal filtering, while the other ones 
correspond to a low-pass temporal filtering). Three stages of decomposition are shown (L and h 
= first stage ; LL and LH = second stage ; LLL and LLH = third stage). 

TTie high fi-equency subbands of each temporal level (H, LH and LLH in the above 
example) and the low frequency subband(s) of the deepest one (LLL) are the spatially analyzed 
through a wavelet filter. An entropy encoder then allows to encode the wavelet coeffidents 
resulting from the spatio-temporal decomposition. I=br ooimple, the 20-SPIHr, originally 
proposed by A. Said and \MA. Pearlman in "A new, fas^ and efficient Image codec based on set 
partitioning in hierarchical trees", IEEE Transactions on QYcufts and Systems for Video 
Technology, vol.6, n^3, June 1996, pp.243-250, has been extended to the 3D wavelet 
decomposition, in order to effidentiy encode the final coeflldent bitplanes with respect to the 
spatio-temporal decomposition structure. 

However, all the 3D subband solutions suffer from the following drawbaclc : since 
an entire GOF Is processed at once, all the pictures In the current GOF have to be stored beibre 
being spatio-temporally analyzed and encoded. The problem is the same at the decoder side, 
where all the frames of a given GOF are decoded together. 

SUMMARY OF THE INVENTION 

It is therefore a first object of the invention to propose a decoding method 
allowing to decrease the high memory demand of the 3D subband approach. 

To this end, the invention relates to a video decoding method such as defined In 
the Introductory part of the description and which is further characterized In that it is iterative 
and comprises as many Iterations as the number of couples of frames in each GOF, each 
iteration itself fnduding, for the reconsh-uctlpn of each successive couple of frames of each GOF, 
the sub-steps of : 

- decoding the part of the coded bilstream that corresponds tx> the current GOF ; 

- from the decoded bitstream thus obtained, storing only the data related to the 
current couple of frames and the appropriate subbands containing some information on at least 
one frame of said current couple of frames ; 

- from said related data and said appropriate subbands, synthesizing the two 
frames of said current couple of frames. 

It is also an object of the invention to propose a decoding device allowing to carry 
out said decoding method, a memory medium induding a code for performing the steps of said 
decoding method, and a corresponding apparatus. 



BRIEF DESCRIPTION OF DRAWINGS 

The present invention will now be described, by way of example, with reference to 
the accompanying drawings in which : 
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- Rg.l fllustrates a 3D subband decomposition, performed In the present case on a 
group of eight frames ; 

- Rg.2 shows, among the subbands obtained by means of said decomposition, the 
subbands tiiat are transmitted and the bitstream thus fbnned ; 

- Figs 3 to 6 illustrate. In the decoding method according to ttie Invention, the 
operations iteratively perfomied for decoding the coded bitstream ; 

- Rg.7 shows an example of a decoding device for the implementation of the 
decoding method according to the Invention. 

DETAILED DESCRIPTION OF THE INVENTION 

As briefly indicated above, the amount of frames that have to be stored at the 
same time when processing a whole GOF is really a problem, and probably the main reason 
that, at that moment^ prevents 3D subband solutions fiom being adopted as standards. For 
Instance, witii a OOF having a typical size of 16 frames, at the decoder side where all the 
frames of the GOF are decoded together, one must be able to decode 16 subbands at the same 
time and additionally to store 16 frames before playing them. Moreover, for real-time playing, 
those 16 frames must be decoded before the frames of the previous GOF are all played. In fac^ 
if N is the number of frames in a GOF and M the minimum number of frames to be played in 
real-time while decoding the next N frames, the decoder needs ( (2 x N) + M ) memory frames 
to be stored at the same time. 

The principle of the invention is to propose a decoding meUiod In which a branch- 
by-branch reconstruction of tiie 3D structure Is performed, instead of a rm)nsbtJcHon of the 
entire tree at once. Less data has to be stored wiUi that technical solution, as it will be shown. 

As illusb-ated in Fig.2 (in the case of a GOF of eight frames.for tixe sake of 
simplicity of the figure), said frames Fl to F8 are grouped into four couples of frames CO to C3. 
At the end of tiie first step of tiie temporal decomposition of tiie original sequence, low 
frequency temporal subbands LO, LI, L2, L3 and high frequency temporal subbands HO, HI, H2, 
H3 are available. While the subbands HO to H3 are coded and transmitted, tiie subbands LO to 
L3 are further decomposed : at tiie end of this second step of tiie decomposition, low frequency 
temporal subbands IXD, LU and high frequency temporal subbands LHO, LHl are available. 
Similarly, while tiie subbands LHO, LHl are coded and transmftted, the subbands LLO, LLl are 
furtfier decomposed and, at tiie end of die tiilrtl step of decomposition (tiie last one In the 
illustrated case), a low frequency temporal subband LLLO and a high frequency temporal 
subband LLHO are available and will be coded and faansmitted. The whole set of transmittfid 
subbands is sun-ounded by a black line in Rg.2. 

it then appears tiiat only the subbands HO, LHO, LLLO and LLHO are needed to 
decode ttie first two frames Fl, F2 (i.e. ttie couple CO) of the GOF. Furttiermore, tiie first 
subband HO contains some information orily on tiiese two first frames F1,F2. So, once these 
frames Fl, F2 are decoded, tiie first subband HO becomes useless and can be deleted and 
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replaced : the next subband HI can be loaded In order tx> decode the next couple CI including 
the two frames F3, F4. Only the subbands HI, LHO, LLLO and LLHO are now needed to decode 
these frames F3, R and, as previously for HO, the subband HI contains some Information only 
on these two frames F3, R. So, once these two frames F3, F4 are decoded, the second 
5 subband HI can be deleted, and replaced by H2. And so on : these operations are repeated for 

ra,F6, F7,l=8/ etc (fn the general case, for all the successive couples of frames of the GOF). The 
bitstream (the illustrated organization of which is only an example that does not limit the scope 
of the invention at the decoding side) thus formed for each successive GOF may be encoded by 
means of an entropy coder followed by an arithmetic coder (for instance referenced 21 and 22 

10 respectively). 

The proposed technical solution is tiien the following. The jsart of the coded 
bitstream corresponding to the current GOF is decoded a first time, but only the coded part 
that. In said bitstream, corresponds to the first couple of frames GO (the two first frames Fl and 
F2) and the subbands HO, LHl, LLLO, LLHO is, in fact, stored and decoded. When the first two 

15 frames F1,F2 have been decoded, the first H subband, referenced HO, becomes useless and its 

memory space can be used for the next subband to be decoded. The coded bitstream Is 
therefore read a second time, In order to decode the second H subband, referenced HI, and the 
next couple of frames CI (F3,F4). When this second decoding step has been performed, said 
subband HI becomes useless and the first LH subband too (referenced UHO). They are 

20 consequentiy deleted and replaced by the next H and LH subbands (respectively referenced H2 

and LHl), that will be obtained thanl<s to a third decoding of the same Input coded bitstream, 
and so on. 

This multipass decoding solution, comprising an iteration per couple of frames in a 
GOF, may be detailed with reference to Rgs 3 to 6. During the first iteration, the coded 
25 bitstream CODB received at the decoding side Is decoded by an arithmetic decoder 31, but only 

the decoded parts corresponding to the first couple of frames CO are stored, i.e. the subbands 
LLLO, LLHO, LHO and HO (see F^.3). With said subbands, the inverse operations, vnth respect to 
those illustrated In Rg.l, are then performed : 

- the decoded subbands LLLO and LLHO are used to synthesize the subband LLO ; 
30 - said synthesized subband LLO and the decoded subband LHO are used to 

synthesize the subband LO ; 

- said synthesized subband LO and the decoded subband HO are used to 
reconstruct the two frames Fl, F2 of the couple of frames CO. 

When this first decoding step is achieved, a second one can begin. The coded 
35 bitstream is read a second time, and only the decoded parts corresponding to the second couple 

of frames CI are now stored : the subbands LLLO, LLHO, LHO and HI (see Rg.4). In fact, the 
dotted information of Fig.4 (LLLO, LLHO, LLO, LHO) can be reused from the first decoding step 
(this is especially true for the bitstream information after the arithmetic decoding, because 
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buffering this compressed information is not really memory consuming). With these subbands, 
the following inverse iterations are now perfomied : 

- the decoded subband LLLO and LLHO are used to synthesize the subband LU) ; 

- said synthesized subband U.0 and the decoded subband LHO are used to 
synthesize the subband LI ; 

- said synthesized subband U and the decoded subband HI are used to 
reconsbuct the two frames F3, R of the couple of frames CI. 

When this second decoding step is achieved, a thinJ one can begin similarly. The 
coded bitstream is read a third time, and only the decoded parts corresponding to the third 
couple of frames C2 are stored : the subbands U±0, LLHO, LHl and H2 (see Rg.5). As 
previously, the dotted infomwtion of Rg.5 (LLLO, LLHO) can be reused from the firet (or second) 
decoding step. The following Inverse operations are performed : 

" *e decoded subbands LLLO and LLHO are used to synthesize the subband LLl ; 

- said synthesized subband LLl and the decoded subband LHl are used to 
synthesize the subband 12 ; 

- said synthesized subband L2 and the decoded subband H2 are used to 
reconstruct the two frames F5, F6 of the couple of frames C2. 

When this third decoding step is achieved, a fourth one can begin similarly. The 
coded bitstream is read a fourth time (the last one for a GOF of four couples of frames), only 
the decoded parts corresponding to the fourth couple of frames C3 being stored : the subbands 
LLLO, LLHO, LHl and H3 (see Rg.6). Similarly, the dotted infomnation of Rg.6 (LLLO, LLHO, LLl, 
LHl) can be reused from the third decoding step. The following inveree operations are 
performed : 

- the decoded subbands LLLO and LLHO are used to synthesize the subband LLl. ; 

- said synthesized subband LLl and the decoded subband LHl are used to 
synthesize the subband L3 ; 

- said synthesized subband L3 and the decoded subband H3 are used to 
reconstruct the two frames F7, F8 of the couple of frames C3. 

This procedure is repeated for all the successive GOfe of the video sequence. 
When decoding the coded bitstream according to this procedure, at most two frames (for 
example Fl, F2) and four subbands (with the same example, HO, LHO, LLHO, LLLO) have to be 
stored at the same time. More generally, if N is the number of frames In a GOF (N = 2" 
preferably), only a limited number of subbands and frames are needed at the same time for 
decoding the bitstream, instead of N subbands and N frames. 

This solution has the main advantage of working in any case, regardless of the 
technique used to implement the encoding method (as nothing has to be changed at the 
encoding side, the solution can be adapted to any 3D subband video decoding technique by 
simply changing the decoder). 
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At the decoding side (or in a server), the corresponding decoding method may be 
implemented in a decoding device such as iilustralBd in Rg.7 and which comprises the following 
main modules. The received coded bitstream is first processed by a decoding device 71, 
comprising for instance in series an arithmetic decoding stage and an entropy decoding stage, 
5 and provided for decoding the coded bitstream including the coded coefficients and the coded 

motion vectors. The decoded coefficients and motion vectors are then received by an inverse 3D 
wavelet transform circuit 72 which Es pro^^ded for reconstructing an output video sequence 
corresponding to the original one. The decoding device may also comprise a resource controller 
73, for verif/ing before each motion vector decoding process the amount of bit budget already 

10 spent and dedding, on the basis of said amount, if the remaining parts of the coded data have 

to be decoded or not 

The foregoing description, presented for purposes of Illustration and description, Is 
not Intended to limit the Invention to the precise form disclosed. Many variations or 
modifications are possible in light of the above teachings and are included within the scope of 

15 the Invention. The encoding and decoding devices may be for instance of the type described in 

the document "A fully scalable 3D subband video codec", V.Bottreau and al.. Proceedings of 
IEEE Conference on Image Processing (laPZOOl), vol.2, pp. 1017-1020, Thessalonild, Greece, 
October 7-10, 2001. 

It may also be understood that the decoding device according to the invention can 

20 be implemented in hardware, software (the coded bitstream being then processed in 

accordance with one or more software programs or codes stored In a memory medium and 
executed by means of a processor in order to reconstruct output frames corresponding to the 
original video sequence), or a combination of software and hardware, without excluding that a 
single item of hardware or software can carry out several functions, or that an assembly of items 

25 of hardware or software or both carry out a single function. The described method and device 

may be Implemented by any type of computer system or other apparatus adapted for carrying 
out the method describe herein. A typical combination of hardware and software could be a 
general-purpose computer system with a computer program that, when loaded and executed, 
controls the computer system such that it carries out the method described herein. A specific 

30 use computer, conteining specialized hardware for carrying out one or more of the functional 

tasks of the invention, could altematlvely be utilized. 

The present invention can also be embedded in a computer program product, 
which comprises all the features enabling the implementation of the method and functions 
described herein, and which - when loaded in a computer system - is able to carry out this 

35 method and these functions. Computer program, software program, program, program product, 

or software, in the present context mean any expression, in any language, code or notation, of 
a set of instructions Intended to cause a system having an information processing capability to 
perform a particular function either directly or after either or both of the following : (a) 
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conversion to another language, code or notation ; and/or (b) reproduction fn a different 
material form. 
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CLAIMS: 

1. A video decxxiing method for the decompression of a coded bitstream 

corresponding to an original video sequence that has been divided into successive git>u|» of 
frames (GOFs) and coded by means of a 3D subband video coding method comprising the 
following steps : 

- a temporal filtering step, perfomned on each successive couple of frames in each 
GOF of said sequence ; 

- a spatial analysis step, performed on said filtered sequence ; 

- an entrc^ coding step, performed on said analyzed, filtered sequence ; 

- an arithmetic coding step, applied to the coded sequence thus obtained and 
delivering said coded bitstream ; 

said decoding method being further charaderized in that It is Iterative and comprises as many 
Iterations as the number of couples of frames in each GOF, each iteration itself including, for the 
reconstruction of each successive couple of frames of each GOF, the sub-steps of : 

- decoding the part: of the coded bitstieam that corresponds to the current GOF ; 

- from the decoded bitstream tiius obtained, storing only the data related to tiie 
current couple of frames and the appropriate subbands containing some infomiation on at least 
one frame of said current couple of frames ; 

- from said related data and said appropriate subbands, syntttesizing the two 
frames of said current couple of frames. 

2. A video decoding method for the decompression of a coded bitstream 

corresponding to an original video sequence that has been divided into successive groups of 
frames (GOFs) and coded by means of a 3D subband video coding meUiod comprising the 
following steps : 

- a motion estimation step, perf'omied on said original sequence ; 

■ a motion compensated temporal filtering step, pertbmied on each successive 
couple of frames in each GOF of said sequence ; 

- a spatial ana^^ls step, peribnned on said filtered sequence ; 

- an enbx>py coding step, peribnned on said analyzed, filtered sequence and on 
motion vectors obtained by means of said motion estimation step ; 

- an arithmetic coding step, applied to ttie coded sequence thus obtained and 
delivering said coded bltsb^m ; 

said decoding method being further characterized In that it Is iterative and comprises as many 
iterations as ttie number of couples of fi^mes in each GOF, each iteration Itself Including, for ttie 
reconshucHon of each successive couple of frames of each GOF, tiie sub-steps of : 

- decoding the part: of the coded bitstream tiiat con-esponds to the current GOF ; ' 

- from the decoded bitstream thus obtained, storing only the date related to the 
current couple of frames and ttie appropriate subbands containing some Infonnation on at least 
one frame of said current couple of frames ; 
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- from said related data and said appropriate subbands, synthesizing the two 
frames of said current cxjuple of frames. 

3. A video decoding device for the decompression of a coded bitstream corresponding 
to an original video sequence that has been divided into successive groups of frames (GOFs) 
and coded by means of a 3D subband video coding method comprising the following steps : 

- a temporal filtering step, performed on each successive couple of frames In each 
GOF of said sequence ; 

- a spatial analysis step, performed on said filtered sequence ; 

- an entropy coding step, performed on said analyzed, filtered sequence ; 

- an arithm^c coding step, applied to the coded sequence thus obtained and 
delivering said coded bitstream ; 

said decoding device being further characterized in that it comprises : 

(1) means for decoding the part of the coded bitstream that corresponds to the 
current GOF ; 

(2) means for storing, from the decoded bitstream thus obtained, only the data 
related to the current coupie of frames and the appropriate subbands containing some 
information on at least one frame of said current couple of frames ; 

(3) means for synthesizing the two frames of said current couple of frames from 
said related data and said appropriate subbands ; 

(4) means for repeating as many times as the number of couples of frames in each 
GOF the successive steps performed by said decoding, storing and synthesizing means. 

4. A video decoding device for the decompression of a coded bitstream corresponding 
to an original video sequence that has been divided into successive groups of frames (GOFs) 
and coded by means of a 3D subbarid video, coding method comprising the following steps : 

a motion estimation step, performed on said original sequence ; 

- a motion compensated temporal filtering step, perfomned on each successive 
couple of frames in each GOF of said sequence ; 

- a spatial analysis step, performed on said filtered sequence ; 

• an entropy coding step, performed on said analyzed, filtered sequence and on 
motion vectors obtained by means of said motion estimation step ; 

an arithmetic coding step, applied to the coded sequence thus obteined and 
delivering said coded bitstream ; 

said decoding device being further characterized in that it comprises : 

(1) means for decoding the part of the coded bitstream that corresponds to the 
current GOF ; 

(2) means for storing, from the decoded bitstream thus obtained, only the date 
related to the current couple of frames and the appropriate subbands containing some 
information on at least one frame of said current coupie of frames ; 
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(3) means for synthesizing the two frames of said current couple of frames from 
said related data and said appropriate subbands ; 

(4) means for repeating as many times as the number of couples of frames In each 
GOF the successive steps performed by said decoding, storing and synthesizing means. 

5. A memory medium Including a computer readable code for the decompression of a 

coded bjtstream corresponding to an original video sequence that has been divided into 
successive groups of frames (GOFs) and coded by means of a 3D subband video coding method 
comprising the following steps : 

- a temporal filtering step -with or without motion compensation- performed on 
each successive couple of frames in each GOF of said sequence ; 

- a spatial analysis step, performed on said filtered sequence ; 

- an entropy coding step, performed on said analyzed, filtered sequence and on 
motion vectors In case of motion compensation ; 

- an arithmetic coding step, applied to the coded sequence thus obtained and 
delivering said coded bitstream ; 

said code comprising : 

- a code for decoding the part of the coded bitstream that corresponds to the 
current GOF ; 

- a code for storing, from the decoded bitstream thus obtained, only the data 
related to the current couple of frames and the appropriate subbands containing some 
information on at least one frame of said current couple of frames ; 

- a code for synthesizing the two frames of said current couple of frames from said 
related data and said appropriate subbands ; 

- a code for repeating as many times as the number of couples of frames in each 
GOF the successive steps performed by said decoding, storing and synthesizing codes. 

6. An apparatus for the decompression of a coded bitstream corresponding to an 

original video sequence that has been divided into successive groups of frames (GOFs) and 
coded by means of a 3D subband video coding method comprising the following steps : 

- a temporal filtering step -with or without motion compensation- performed on 
each successive couple of frames in each GOF of said sequence ; 

- a spatial analysis step, perfbmried on said filtered sequence ; 

- an entropy coding step, peri^ormed on said analyzed, filtered sequence and on 
motion vectors In case of motion compensation ; 

- an arithmetic ceding step, applied to the coded sequence thus obtained and 
delivering said coded bitstream ; 

said apparatus comprising a memory which stores executable code and a processor which 
executes the code stored in the memory so as to : 

- decode the part of said bitstream that corresponds to the current GOF ; 
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- Store, from the decoded bitstream thus obtain^, only the data related to the 
current couple of franaes and the appropriate subbands containing some information on at least 
one frame of said current couple of frames ; 

- synthesize the two frames of said current couple of frames from said related data 
and said appropriate subbands ; 

- repeat as many times as the number of couples of frames in each GOF these 
decoding, storing and synthesizing operations applied to the current couple of frames. 
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Abstract 

The invention relates to a video decoding method for tJie decompr^ion of a 
coded bitstream corresponding to an original video sequence. The sequence has been divided 
into successive groups of frames (GOFs) and coded by means of a 3D subband video coding 
method. According to the invention, the decoding method is Iterative and comprises as many 
iterations as the number of couples of frames In each GOF, each Iteration itself including, for the 
reconstruction of each successive couple of frames of each GOF, the sub-steps of decoding the 
part of tiie coded bitstream tiiat corresponds to the current GOF, storing, from the decoded 
bitsb-eam thus obtained, only the data related to the current couple of frames and appropriate 
subbands containing some Infomtation on at least one frame of said cun-ent couple of flames, 
and, frmn said related data and »ld appropriate subbands, synthesizing the h«o frames of said 
current couple of frames. 
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